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ABSTRACT 

This study examined the validity and reliability of a newly 
developed instrument for performance assessment of teachers. The instrument 
was based on the "framework for teaching" proposed by C. Danielson, a 
framework that identifies the aspects of a teacher's responsibilities that 
have been documented through empirical studies and theoretical research as 
promoting student learning. The University Supervisor's Evaluation Report 
(USER) was developed based on 22 components in the 4 domains of Danielson's 
framework. Two slightly modified instruments, the Student's Self-Report and 
the Cooperating Teacher's Evaluation Report (CTER) , were developed for the 
purpose of validation. A panel of 23 members, including students, cooperating 
teachers, and university supervisors, was asked to evaluate the contents of 
each instrument in relation to the four domains of Danielson's framework. 
Thirty-three complete data sets were collected from student teachers and 
their cooperating teachers and university supervisors. In general, the 
internal consistency reliability within each domain was reasonably high 
except for domain 4 of the CTER. The interrater reliability was also high, as 
scores among students, cooperating teachers, and university supervisors were 
consistent. Some items, however, were deemed not suitable after careful 
examination, and these may be eliminated from the instrument. Overall, the 
new instrument was found to have high content validity and construct 
validity, but the concurrent validity comparing the existing assessment tool 
with the newly developed instrument was low. The existing instrument had 
never been validated, and may* not have been appropriate for evaluating 
student teacher competence. Appendix A contains Danielson's framework, and 
Appendix B contains the instrument. (SLD) 
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Development and Validation of Student Teaching Performance Assessment 
Based on Danielson's Framework for Teaching 
Introduction 

The educational goal of the Department of Education and Special Education at a rural 

\ 

university in Pennsylvania is to prepare effective teachers. As American society becomes a 
technology-based culture based on information and communication, requirements for the 
preparation and certification of teachers are changing from a time-based, course-completion 
model of teacher education to one with performance-based standards. More emphasis is being 
placed on the quality of teacher candidates' work, their subject matter knowledge, and their 
demonstrated teaching skills (Brucklacher, 1999). This rural Pennsylvania state university has 
responded to such increasing demands, by tailoring their teacher education programs to ensure 
that all pre-service possess a strong foundation of content knowledge, as well as the ability to 
apply their knowledge in authentic teaching situations. 

Both the National Council for Accreditation of Teacher Education (NCATE) and the 
Pennsylvania Department of Education (PDE) adopted new standards that require all member 
universities to carefully assess the performance of their candidates and graduates 
(http://www.pde.psu.edu/ & http://www.ncate.org/). In the revised standards, program evaluation 
will be driven by the quality of the assessments used. Both NCATE and PDE emphasize and 
reinforce performance-based outcomes for prospective teachers and recommend a program focus 
on performance in the classrooms of the public schools. 

In order to meet changing NCATE and PDE standards, our Education department is in 
the process of implementing changes to the curriculum based on "the framework for teaching" 
proposed by Charlotte Danielson. This framework for teaching identifies those aspects of a 
teacher's responsibilities that have been documented through empirical studies and theoretical 
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research as promoting improved student learning (Danielson, 1996). For example: The Scientific 
Basis of the Art of Teaching (Gage, 1 977) and Handbook of Research on Teaching (Wittrock, 
1986). Those responsibilities seek to define what teachers should know and be able to do in the 
exercise of their profession. In this framework, the complex activity of teaching is divided into 
22 components clustered into four domains of teaching responsibility: 1) planning and 
preparation, 2) classroom environment, 3) instruction, and 4) professional responsibilities 
(Danielson, 1996) (See Appendix A). 

Given the changing nature of goals in education, the imperfect nature of assessment, and 
the difficulties of social research, few studies that yield definitive answers to what instructional 
approaches maximize student learning exist (Danielson, 1996). Therefore, this “framework for 
teaching” possesses limitations resulting from the disparities that exist in the educational 
community regarding educational goals, assessment, and control of extraneous variables in 
educational research. 

Firstly, appropriate educational goals are difficult to define. Policy leaders and educators 
present an argument that the goals and standards for education over the past 1 00 years have not 
been rigorous enough (National Board for Professional Teaching Standards, 1991). Current state 
and district goals are attempts to respond to the raising standards and increasing demands. But, 
members of the educational community still have not reached a consensus on what educational 
goals will be most effective in preparing students for the challenges of today and tomorrow. 

Secondly, assessment measures currently used in educational research are inadequate for 
determining the comparative effectiveness of different approaches to teaching and learning. 
Valid and reliable assessments are urgently needed for the new generation of instructional goals. 
Until researchers have these measures, they cannot conduct credible research on the 
effectiveness of alternative approaches. 
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Thirdly, extraneous variables in educational, as well as social research, are difficult to 
control. For a clean research design, both the students and the teachers involved in the research 
and assigned to the experimental and control groups must be comparable. Difficulties are evident 
when one considers the fact that the random assignment of teachers will indefinitely place some 
teachers in groups that require the use of approaches they do not support. Extraneous variables 
such as these complicate a determination of what approaches are indeed most effective. 

Despite these circumstantial limitations, Danielson’s “framework for teaching” provides 
support for teachers, by guiding their on-going efforts to provide students with the academic - 
knowledge and social faculties necessary for current and future success. In addition, a new 
paradigm for learning and teaching has been developed to meet the demands of the present and 
future. Traditional views of learning focused on knowledge and procedures of low cognitive 
challenge and the regurgitation of superficial understanding. Gradually, this view has shifted to a 
constructivist approach of learning (and teaching). 

Recently, educators and policymakers have been speaking of the philosophy of 
contructivism and a constructivist approach to teaching and learning. But, this movement is not 
new. Since the early 20 th century, the cognitive approaches rooted in the philosophy of 
constructivism have competed for attention with behaviorist orientations (Danielson, 1996). The 
philosophy of contructivism supports the notion that people’s understanding of any concept 
depends entirely on their mental construction of that concept (Danielson, 1996). Although 
teachers can guide the learning process, students must assume the responsibility of managing the 
process and developing an understanding for themselves. 

In response to the changing NCATE and PDE Standards, Charlotte Danielson’s 
“framework for teaching” was adopted and the constructivist philosophy of learning (and 
teaching) was implemented throughout all courses and field experiences in the Education 
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program at this University. To evaluate the effectiveness of the program under a curriculum 
change, a reliable and valid assessment instrument is essential. The purpose of this study is to 
newly develop and validate a performance-based instrument. 

Method 

Development of Instrument 

A "University Supervisor's Evaluation Report (USER)" was developed based on 22 
components within the 4 domains of Danielson's framework. Within each component, several 
related statements were indicated to provide a more objective rating criterion. This instrument is 
using a 4-point Likert Scale, ranging from least competent to highly competent, and consists of 
22 items within 4 domains. In addition, two slightly modified instruments, namely, "Student's 
Self-Report" (SSR) and "Cooperating Teacher's Evaluation Report" (CTER), were also 
developed for the purpose of validation. (See Appendix B) 

Procedures 

A panel of 23 members including students, cooperating teachers and university 
supervisors were asked to evaluate the contents of each instrument in relation to the four 
domains of Danielson’s framework. Data was collected to examine content validity. A total of 56 
student teachers, enrolled in the Elementary Education, Secondary Education, and Special 
Education programs, were recruited. At the end of student teaching, the students, cooperating 
teachers, and university supervisors completed the SSR, CTER and USER respectively, in 
regards to the student teacher’s competency. A total of 33 complete data sets were collected at 
the end of the student teaching semester. 

Analysis 

The validity and reliability of the 3 instruments were tested. Content validity was 
determined from the percent agreement across a panel of 23 experts, who were asked to judge 
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whether or not the statements were valid measures of the specific components. Concurrent 
validity was established by correlating the ratings on the USER and the student teachers’ grades 
(GRADES), assuming that the student teaching grade was valid. Additional analyses also 
examined the correlation between the USER and student Quality Point Average (QPA), and the 
correlation between GRADES and QPA. Confirmatory factor analysis using Linear Structural 
Relations (LISREL) statistical software was performed to examine the construct validity of the 
USER. Convergent and discriminant validity using multi-trait multi-method (3 instruments, 4 
domains) was also examined. Internal consistency reliability using SPSS statistical software was 
performed to examine the instrument reliability. Inter-rater reliability was determined by the 
score consistency between USER and CTER, USER and SSR, and CTER and SSR. 

Results 

Content validity of 96% agreement was found for the USER instrument (N=23). (See 
Table I). The correlation between the USER and the student teaching grade (GRADES) is 
significant (N=33, r=.453, p=.02). No significant correlation was found between the USER and 
QPA (r=.077, p=,676), or between GRADES and QPA (r=-.123, p=.558). 

Confirmatory factor analysis indicated that the model fit (the instrument contained 4 
domains, normed fit index (NFI) = .89, comparative fit index (CFI) = 1.00, goodness fit index 
(GFI) = .91). Table II shows the multi-trait multi-method correlation matrix comparing the 
USER, CTER, and SSR on all four domains. It also shows internal consistency reliability of all 
three instruments. The convergent validity is higher than the discriminant validity except for 
domain 2. The correlations within each domain were higher than .84 for the USER and the 
CTER, but lower for the SSR. All of the convergent correlation coefficients were significant 

(p<01). 

Internal consistency reliability of the USER for each domain ranged from .8384 (N=32 
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Domain 2) to .9310 (N=20, Domain 4), with Domain 1 (N=33, r=.9097) and Domain3 (N=32, 
r=.8514) in the middle. No significant differences were found between the SSR and USER, SSR 
and CTER, or USER and CTER, which indicates that students' self evaluations were consistent 
with their cooperating teachers' and university supervisors' evaluations. 

Discussion 

This study examined the validity and the reliability of a newly developed instrument for 
performance assessment of student teachers. In general, the internal consistency reliability within 
each domain was reasonably high except for the CTER, domain 4. The inter-rater reliability was 
also high, as the scores among students, cooperating teachers, and university supervisors were 
consistent. However, after carefully examining each item, some items were not suitable. For 
example, item 4C "Communicating with families" was not applicable to our student teachers. 
Perhaps this item should be eliminated. 

The new instrument was found to have high content validity and construct validity. 
However, the concurrent validity comparing the existing assessment tool with the newly 
developed instrument was low. The existing instrument was never validated, and thus may not be 
appropriate for evaluating student teacher competency. Another interesting finding revealed low 
correlation between the USER and QPA. This may suggest that students who know content 
(high QPA) do not necessarily know how to teach. 

This study presents certain limitations. Since this study is preliminary, a small number 
subjects were recruited. The preliminary analyses only provided limited information about the 
instrument. A large data set is necessary for the validation of the instrument. Further research 
should be done in revising items and recruiting more subjects. 

As the standards of NCATE and PDE change towards a more performance-based 
assessment, a few universities have initiated the development of assessment tools based on 
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Danielson's framework, (e.g . Slippery Rock University). However, these instruments have not 
been quantitatively validated. The validation of this instrument not only provides a valid and 
reliable assessment of our student teachers, but also provides an example for other teacher 
preparation institutions, as a partial reference for their program evaluation efforts. 
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Table I 

Content Validity of University Supervisor 's Evaluation Report (N=23) 



Domain 


Component 


Agree (%) 


Disagree (%) 


1 . Planning and Preparation 


3,. knowledge of content and pedagogy 


100 


0 


l. 


b. demonstrating knowledge of students 


100 


0 


l. 


C. selecting instructional goals 


100 


0 


l. 


d. knowledge of resources 


91.3 


8.7 


l. 


6. designing coherent instruction 


100 


0 


l. 


f. assessing student learning 


100 


0 


2. The Classroom Environment 


a. environment of respect and rapport 


95.65 


4.35 


2. 


b. establishing a culture for learning 


95.65 


4.35 


2. 


C. managing classroom procedures 


91.3 


8.7 


2. 


d. managing student behavior 


95.65 


4.35 


2. 


6. organizing physical space 


86.96 


13.04 


3. Instruction 


a. coinmuni eating clearly and accurately 


100 


0 


3. 


b. questioning and discussion techniques 


100 


0 


3. 


C. engaging students in learning 


95.65 


4.35 


3. 


d. providing feedback to students 


100 


0 


3. 


C. flexibility and responsiveness 


100 


0 


4. Professional Responsibilities 


a. reflecting on teaching 


100 


0 


4. 


b. maintaining accurate records 


100 


0 


4. 


C. communicating with families 


82.61 


17.39 


4. 


d. contributing to the school and district 


91.3 


8.7 


4. 


6. growing and developing professionally 


95.65 


4.35 


4. 


f. showing professionalism 


95.65 


4.35 
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Table II 

Multi-Trait Multi-Method Correlation Matrix Comparing USER with CTER & SSR on 



All Four Domains and the ' 


nterna 


Consistency 


Reliabil 


ity 




SU1 


SU2 


SU3 


SU4 


SSI 


SS2 


SS3 


SS4 


CT1 


CT2 


CT3 


CT4 


SU1 


(.91) 
























SU2 


.83 


(.84) 






















SU3 


.85 


.80 


(.85) 




















SU4 


.90 


.89 


.82 


(.93) 


















SSI 


.62 


.55 


.55 


.60 


(.77) 
















SS2 


.49 


.45 


.29 


.54 


.68 


(.85) 














SS3 


.54 


.38 


.51 


.51 


.76 


.70 


(.73) 












SS4 


.49 


.53 


.47 


.47 


.81 


.66 


.64 


(.75) 










CT1 


.74 


.64 


.71 


.68 


.69 


.68 


.73 


.68 


(.89) 








CT2 


.49 


.47 


.43 


.50 


.58 


.80 


.69 


.70 


.85 


(.85) 






CT3 


.60 


.54 


.70 


.54 


.51 


.50 


.66 


.49 


.91 


.80 


(.86) 




CT4 


.64 


.58 


.60 


.68 


.72 


.74 


.80 


.68 


.84 


.83 


.77 


(.27) 



All convergent validity is significant (p<.01, two tails) 
SU: Supervisor's Report; 

SS: Student Self Report 

CT: Cooperating Teacher's Report 

Numbers: indicate domains 

Internal consistency reliability is indicated in ( ) 

Convergent validity is indicated in bold 
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APPENDIX A 

Charlotte Danielson’s “framework for teaching” 
Components of professional practice 



Domain 1: Planning. Preparation, and 

Assessment 

la: Demonstrating knowledge of 
content and pedagogy 

lb: Demonstrating knowledge of 
students 

lc: Selecting instructional goals 
and objectives 

Id: Demonstrating knowledge of 
resources for teaching and for 
students' use 

le: Designing coherent instruction 

If: Assessing student learning 

Domain 3: Instruction and 

Communication 

3a: Communicating clearly and 
accurately 

3b: Using questioning and discussion 
techniques 

3c: Engaging students in learning 

3d: Providing feedback to students 

3e: Demonstrating flexibility and 
responsiveness 



Domain 2: The Classroom 
Environment 

2a: Creating an environment of respect 
and rapport 

2b: Establishing a culture for learning 

2c: Managing classroom procedures 

2d: Managing student behavior 

2e: Organizing physical space 

Domain 4: Professional Responsibility 

4a: Reflecting on teaching 

4b: Maintaining accurate records 

4c: Communicating with families 

4d: Working in and contributing to the 
school and district 

4e: Growing and developing 
professionally 

4f: Showing professionalism 
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APPENDIX B 

FRAMEWORK FOR TEACHING 
SUPERVISOR'S EVALUATION OF STUDENT TEACHER 

Student Teacher ID# 

Month/Day/Year Grade/Subject 



As the student teacher’s advisor, please rate your student teacher’s performance by circling the 
number that best characterizes his/her abilities in each component. Use the following scale: 

(weak evidence) 1 - 2-3-4 (strong evidence) 

Feel free to write co mm ents in the space available to the right. 

Please rate your student teacher’s performance by circling the number that best characterizes his/her 
performance in each component. Use the following scale: (weak evidence) 1 -2-3-4 (strong evidence) 
Feel free to write comments in the space available to the right. It is your discretion whether the item is 
applicable to your student teacher. Please circle NA if you believe the item is not applicable. In 
addition, some of the statements under each heading may not be applicable while others are, student 
teacher need not demonstrate all of them to receive a "4". Once you complete your evaluation, please 
total your score in the box provided. Thank you. 



Domain 1: Planning, Preparation, and Assessment 



Components: 

la: Demonstrating Knowledge of Content and Pedagogy 1 2 3 4 NA 

• Applies accepted learning principles 

• Prepares for and knows assigned content 

• Writes lesson plans that contain accurate information 

• Identifies common student misconceptions and likely sources of error 

• Selects teaching techniques appropriate to students’ ability levels and the content 

• Makes reference to relevant school, district, state and national standards in lesson plans 

• Answers student questions with accurate information 

• Communicates content in ways students can understand 

lb: Demonstrating Knowledge of Students 1 2 3 4 NA 

• Employs principles of human growth and development 

• Shows understanding of and sensitivity to community and cultural norms 

® Shows sensitivity to student needs 

• Becomes familiar with what students know and can do 

• Becomes familiar with what interests students 

• Values the resources individual students bring to the classroom 

lc: Selecting Instructional Goals and Objectives 1 2 3 4 NA 

• Analyzes needs of diverse learners 

• Identifies important concepts and skills 

• Writes clear instructional goals and expressive objectives 

• Matches goals and objectives to students’ ability levels 

• Writes goals and objectives that correspond to viable methods of assessment 

• Provides a rationale for planning and instructional decisions 
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Id: Demonstrating Knowledge of Resources for Teaching and for 12 3 4 

Students * Use 

• Uses school and district resources 

• Uses technology to locate and obtain resources 

• Visits libraries and other community organizations for resources 

• Is willing to find non- traditional resources outside immediate district or community 

le: Designing Coherent Instruction 12 3 4 

Plans thoroughly and consistently 

Meets students’ current needs and leads into the next level of learning 
Selects appropriate instructional groups based on students' abilities and needs 
Uses a variety of instructional techniques and materials 

Designs activities that represent relevant and authentic applications of knowledge 
Incorporates students’ interests into lessons 
Organizes necessary materials, supplies, and equipment 
Develops long-term instructional sequences 

If: Assessing Student Learning 12 3 4 

Selects or designs appropriate formal/ informal assessment materials 
Adjusts assessment methods for varying students’ needs and abilities 
Articulates congruence of assessment methods with instructional goals and objectives 
Clearly communicates criteria and standards to students 
Uses assessment results in planning subsequent instruction 
Documents student progress 
U tiliz es techniques for student self-assessment 



NA 



NA 



NA 



Domain 2: The Classroom Environment 



Components: 

2a: Creating an Environment of Respect and Rapport 12 3 4 

Respects and values individual differences 
Relates positively to students (e.g. warmth, caring, respect) 

Shows concern for students 
Establishes appropriate rapport with students 

Encourages responses and interactions that foster positive (e.g. polite and respectful 
relationships among students 



NA 



2b: Establishing a Culture for Learning 

• Identifies factors that affect students’ learning and behavior 

• Cultivates students’ interest in the value of the content 

• Instills students' pride in learning and in their work 

• Provides students with opportunities for optimal learning 



2 3 4 NA 



2c: Managing Classroom Procedures 

• Follows daily routines for managing instructional groups 

• Makes smooth transitions with little loss of instructional time 

• Follows efficient routines for handling materials and supplies 

• Uses effective strategies for managing non-instructional duties 

• Supervises volunteers and paraprofessionals appropriately 



1 2 3 4 NA 
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2d: Managing Student Behavior 1 

• Observes and monitors student behavior 

• Assists students to apply conflict resolution techniques 

• Monitors the interactions of instructional groups to ma ximi ze productivity 

• Responds appropriately to student behavior 

• Works with students to encourage positive behavior choices 

• Encourages student self-discipline 

• Responds appropriately and successfully to student misbehavior 



3 4 NA 



2e: Organizing Physical Space 1 2 3 4 NA 

• Creates and maintains a physical environment that is safe 

• Develops functional arrangements conducive to learning 

• Arranges space so all students have access to learning 



Domain 3: Instruction and Communication 



Components: 

3a: Communicating Clearly and Accurately 1 2 3 4 NA 

• Uses appropriate and expressive vocabulary 

• Spells correctly 

• Writes legibly 

• Uses voice effectively through proper enunciation, volume, pitch, and speed 

• Uses proper /acceptable English grammar and syntax 

• Presents ideas clearly and concisely 

• Expresses ideas accurately and logically 

• Utilizes appropriate eye contact, posture, and other nonverbal gestures 

• Communicates equitably across various subgroups of students 

3b: Using Questioning and Discussion Techniques 1 2 3 4 NA 

• Uses questions at various cognitive levels to encourage higher levels of thinking 

• Uses open-ended questions 

• Uses questions that encourage exploration of content 

• Uses adequate wait time after asking questions and after students respond 

• Probes student responses, seeks clarification or elaboration 

• Engages a wide range of students in discussion 

3c: Engaging Students in Learning 1 2 3 4 NA 

• Connects lesson content to students’ knowledge, interests, experiences, and culture 

• Communicates goals, objectives, directions, and procedures clearly 

• Selects examples and metaphors that illustrate ideas and skills 

• Paces lessons appropriately based on all students’ needs and abilities 

• Incorporates a variety of activities and materials suitable to instructional goals and objectives 

• Encourages /Includes problem-based learning 

• Permits student choice and initiative 



3d: Providing Feedback to Students 1 2 3 4 NA 

• Provides accurate, specific, substantive and constructive feedback that advances 
understanding 

• Meaningfully evaluates student work and returns it in a timely manner 

• Gives feedback in a timely manner 

• Sensitive to verbal and nonverbal signals from students that indicate lack of understanding 

• Handles incorrect or misleading student responses in a manner that encourages learning 
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3e: Demonstrating Flexibility and Responsiveness 1 2 3 4 NA 

• Adapts instruction to meet the learning needs of all students 

• Revises activities during instruction in response to student feedback 

• Adjusts instruction to use teachable moments 

• Persists in seeking to help students who are disengaged or who are having difficulty learning 



Exhibits initiative, originality, and creativity 

Reflects upon teaching and continually refines instruction 



Domain 4: Professional Responsibilities 



Components: 

4a: Reflecting on Teaching 1 2 3 4 NA 

• Articulates an understanding of lessons’ goals and objectives 

• States the strengths and weaknesses of lessons 

• Analyzes the results of students’ participation __ 

• Evaluates instructional sequence 

• Supports judgments with data related to lessons 

• Writes reflections about lessons 

• Develops written plans for improvement 

• Accepts feedback and implements recommendations 

4b: Maintaining Accurate Records 1 2 3 4 NA 

• Records accurately and maintains the results of student’s assignments 

• Records systematically information about student progress in learning 

• Keeps complete records about non-instructional activities and students contribute to its 
maintenance 

• Respects test and test score security issues/ confidentiality 



4c: Communicating with Families 1 2 3 4 NA 

• Follows proper procedures regarding confidentiality 

• Communicates positive information and concerns to parent/guardians 

• Participates in parent- teacher conferences 

4d: Working in and Contributing to the School and District 1 2 3 4 NA 

• Works well with others 

• Seeks help from other professionals concerning teaching and learning 

• Volunteers to participate in school-related and community activities making a substantial 
contribution 



4e: Growing and Developing Professionally 1 2 3 4 NA 

• Participates in student teaching seminars and other required university events 

• Attends school and district professional development programs 

• Reads professional journals and other teaching related literature 

• Shows commitment to professional growth and development 

• Articulates a personal philosophy of education that reflects growing self-awareness 
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4f: Showing Professionalism 12 3 

• Attends promptly and regularly 

• Completes schedules, assignments, and other paperwork on time 

• Completes work in the manner prescribed 

• Dresses commensurate with professional responsibilities 

• Has neat grooming and practices good personal hygiene 

• Follows school and class rules 

• Follows relevant codes of ethics for the teaching profession 

• Follows proper procedures for reporting students' welfare 

• Shows respect for school and personal property 

• Makes an effort to challenge prejudicial attitudes 
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